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Description 

Background of the Invention 

This invention relates to synchronization of various 
databases in a network. More particularly, the invention 
relates to synchronization of replicated data across het- 
erogeneous databases in a communication network. 

Many types of databases are known. There are re- 
lational model database, hierarchical model database, 
network model database, and many other database 
models. Among the available database models, the re- 
lational model is the most popular model. 

In the relational database model, the database is 
divided into two areas - physical model and logical mod- 
el where the physical model is based on each entity and 
the logical model is based on entity relationship. The 
physical and logical models are then used to generate 
a physical structure of a relational model database. 

The hierarchical model has data structured in a par- 
ent-child relationship where the data of the child can be 
accessed only through the parent. Thus, in hierarchical 
database, the access to a child entity dependent on a 
parent entity can be only accessed if the key leading to 
the parent entity is available. In contrast, in the relational 
model, the data of any entity can be accessed as long 
as there is an appropriate key leading to such database. 

For example, one way to access the relational da- 
tabase is a well-known definer query language called 
Structural Query Language ("SQL" or Sequel). But for 
the hierarchical database, since there is no relationship 
to connect the entities, it requires a very specific function 
call to access each entity by giving the segment key or 
by giving some other type of identifier to access the par- 
ent of the entity. Once the parent is accessed, the child 
segment can be searched or a segment number can be 
directly called on for locating the entity. If the entity is 
still in another child segment, the searching continues 
down to the deeper level until the entity is located. 

The network model is similar to the relational model. 
The entities are all related where the relationship be- 
tween the segments is called owner and membership. 
The owner owns many members and each member 
could be the owner of other members. 

Other database models include proprietary data- 
base model and object-oriented database model. The 
proprietary database model can be of any construction 
that is suited to the use of a particular software vendor. 
Thus, the proprietary model may have any of combined 
features derived relational, hierarchical and network 
models or may have a totally different structure from any 
of the traditional database models. 

Since significant amount of replicated information 
is resident among the network elements, operation sys- 
tems, billing and support systems; it is important that the 
replicated copies of primary data are synchronized with 
the primary data. However, keeping consistent replicat- 
ed information becomes more difficult where network el- 



ements can be any type of the database models dis- 
cussed above such as the relational model, hierarchical 
model or network model. 

In the telecommunication industry, a standard inter- 

5 face called Open Database Connectivity ("ODBC") is 
used for communication among different types of plat- 
forms such as IBM mainframes and Unix workstations. 
However, ODBC is not useful if the databases of these 
different platforms are different type. For example, when 

10 the primary database is relational and the subscribing 
databases are hierarchical, the updated data in the pri- 
mary database may not be propagated because the pri- 
mary database uses Structural Query Language 
("SQL") while the subscribing databases do not recog- 

15 nize SQL. 

The data inconsistency among these database 
models, if occurs, not only causes a significant revenue 
loss from a business standpoint, but also requires ex- 
tremely high cost manual data synchronization. Further, 

20 the manual data synchronization involving human serv- 
ice may cause more data inconsistency problems. 

Currently available systems for updating subscrib- 
ing databases are designed for homogenous databas- 
es. The current systems also use the protocol called 

25 two-face commit. " The two-face commit means that the 
updating of subscribing databases occur only when all 
of the subscribing databases can be updated. Although 
this type of system provides that all subscribing data- 
bases at all times contain same information, the data- 

30 base updates may not be propagated to the databases 
that need the updates due to the failed acknowledgment 
from the databases that do not need the updates. Also, 
if a single subscribing database fails to acknowledge an 
update availability, it prevents all the rest of the subscrib- 
es ing databases from receiving the database updates. 

It is therefore an object of this invention to provide 
an efficient system and method that accurately and 
promptly synchronize heterogeneous databases in a 
network. 

40 it is also an object of this invention to provide a sys- 
tem and method that correctly propagates the updates 
in the primary data only to subscribing databases that 
need the updates. 

45 Summary of the Invention 

These and other objects of the invention are accom- 
plished in accordance with the principles of the invention 
by providing a database network having a primary da- 

50 tabase and a plurality of heterogeneous subscribing da- 
tabases for replicating data updates of the primary da- 
tabase in the heterogeneous subscribing databases. 
The database network includes a primary database en- 
gine connected to the primary database for capturing 

55 the data updates in the primary database. A query man- 
ager connected to the database engine for generating 
queries translates queries based on a specified format 
for each of the heterogeneous subscribing databases. 
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A data distributor connected to the query manager dis- 
tributes the translated queries to the heterogeneous 
subscribing databases. 

The database network may further include a sub- 
scription controller for verifying that at least one of said 
subscribing databases needs the updates made in the 
primary database so that the query manager generates 
queries only for the subscribing databases that need the 
updates. 

The database network may also include a synchro- 
nization manager for controlling sequences of the queue 
distribution where each queue is categorized as imme- 
diate or deferred queue based on query's time sensitiv- 
ity. The queue may be categorized as a retry queue if 
the previous transmission has failed. 

Further features of the invention, its nature and var- 
ious advantages will be more apparent from the accom- 
panying drawings and the following detailed description 
of the preferred embodiments. 

Brief Description of the Drawings 

FIG. 1 is a simplified block diagram of illustrative 
network which can be operated in accordance with this 
invention. 

FIG. 2 is a more detailed block diagram of illustra- 
tive integrated data consistency server of the network 
of FIG. 1. 

FIGS. S-5 are a flow chart of steps for carrying out 
an illustrative embodiment of the method of this inven- 
tion. 

Detailed Description of the Preferred Embodiments 

In the illustrative embodiment shown in FIG. 1 , da- 
tabase network 100 of the present invention includes 
primary system 102 connected to subscribing systems 
104 via communication network 106. 

Primary system 102 contains primary application 
108, primary database engine 110, primary database 
1 1 2 and integrated data consistency server 11 4. Primary 
database engine 11 0 detects the transactions in primary 
application 108 that causes updates in primary data- 
base 112 and captures the updates to forward to inte- 
grated consistency server 114. Upon the receipt of the 
updates, integrated consistency server 114 runs the 
steps illustrated in FIGs. 3 and 4, while the update da- 
tabase transactions are performed at primary database 
112. 

Communication network 106 delivers the proc- 
essed results from integrated data consistency server 
114 to proper database engines 116, 118 and 120 of 
subscribing systems 104. For example, subscribing sys- 
tems 104 include database engines 116, 118 and 120 
attached to hierarchical database 122, relational data- 
base 124 and network database 126. Many other types 
of databases can be added to subscribing systems 1 04. 
Database engine 116, 118 and 120 each receives ap- 



propriately formatted database updates and updates hi- 
erarchical database 122, relational database 124 and 
network database 126 respectively. 

FIG. 2 shows integrated data consistency server 

5 114 (FIG. 1 ) connected to primary database engine 110 
(FIG. 1) and communication network 106 (FIG. 1) in 
more details. Integrated data consistency server 11 4 in- 
cludes subscription controller ("SC") 200, query manag- 
er COM") 204, update queues ("Un") 206, queue con- 

10 trailers ('Qc') 208, concurrence controller fCC") 210, 
data distributor ("DD") 21 2 and response handler ("RH") 
214. 

Subscription controller 200 is connected to data- 
base engine 110 and receives the data updates cap- 
tured or recorded by primary database engine 110. 
Based on the captured data updates, subscription con- 
troller 200 determines whether the data updates need 
to be propagated at all. The data updates need to be 
propagated if any of databases 1 22, 1 24 and 1 26 needs 

20 the data updates for its application. 

Based on the findings of subscription controller 200, 
query manager 204 connected to subscription controller 
200 generates queries only for the databases that need 
the updates. Query manager 200 then determines 

25 which of subscribing databases requires translations 
and subsequently translates the queries into appropri- 
ate formats based on the types of subscribing database 
models. Query manager 202 is connected to synchro- 
nization manager 204 so that when a query is generat- 

30 ed, a notice alerts synchronization manager 204. 

Update queues 206 connected to both query man- 
ager 204 and synchronization manager 204 store the 
queries generated from query manager 204. Each 
queue also contains the number of update transactions 

as ('Un') which indicate the total update transactions 
stored in each update queue. 

Each of queue controller 208 is connected to each 
of corresponding update queues 206 and synchroniza- 
tion manager 204. Thus, the number of queuing control- 

40 lers indicate the number of present queues waiting for 
transmission to subscribing system 104. 

Concurrence controller 210 connected to queue 
controller 208 and synchronization manager 204 con- 
trols the openings of the distribution channel of data dis- 

45 tributor 212 and contains the control level ("CI") infor- 
mation which indicate the number of available distribu- 
tion channels. 

Data distributor 21 2 is connected to both concurrent 
controller 210 and synchronization manager 204 distrib- 

50 utes updates in queues to subscribing systems 104. 

Communication network 106 connects data distrib- 
utor 212 and subscribing systems 104. Communication 
network 1 06 may contain various types of communica- 
tion means such as underground telecommunication 

55 links, wireless links and satellite links. Communication 
network 1 06 may also contain subnetwork such as com- 
puter network or cable television network. 

The functions of subscription controller 200, query 
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manager 204, update queue 206, queue controller 208, 
concurrence controller 21 0, data distributor 21 2 and re- 
sponse handler 214 are also illustrated with respect to 
steps in FIGs. 3 and 4. 

FIG. 3 shows an illustrative sequence of steps in 
accordance with this invention for updating the subscrib- 
ing systems 104 (FIG. 1) with the data updates made in 
primary system 102 (FIG. 1 ). However, the order of the 
steps is not critical and can be arranged in various other 
ways without compromising the operation of the heter- 
ogeneous database network 100 (FIG. 1) of present in- 
vention. 

In step 300, primary application 108 (FIG. 1) proc- 
esses transactions that require database updates in pri- 
mary database 112 (FIG. 1). 

In step 302, primary database engine 110 (FIG. 1) 
detects the database update transactions and captures 
the data updates to be made in primary database 112 
(FIG. 1). 

In step 304, primary database engine 110 passes 
the captured data updates to subscriber controller 200 
(FIG. 2) while primary database 112 (FIG. 1) is updated. 

In step 306, subscriber controller 200 (FIG. 2) de- 
termines subscription of subscribing systems 104 (FIG. 
1 ), i.e. whether any database in subscription systems 
1 04 (FIG. 1 ) is affected by the update transactions. Sub- 
scriber controller 200 (FIG. 2) thus determines if any of 
subscribing databases 122, 124 and 1 26 needs the up- 
dates that are made in primary database 112 (FIG. 1). 

If none of subscribing databases 122, 124 and 126 
(FIG. 1 ) needs the updates, integrated data consistency 
server 1 1 4 (FIG. 1 ) becomes idle and returns to step 302 
to wait for the detection of database engine 110 (FIG. 
1 ) for next update transaction in primary system 1 02 
(FIG. 1). 

If any subscribing databases 122, 124 and 126 
(FIG. 1) needs the updates of primary database 112 
(FIG. 1), then query manager 202 (FIG. 2) in step 308 
receives the updates. 

In step 310, query manager 202 (FIG. 2) retrieves 
registration data from primary database 112 (FIG. 1) or, 
alternatively, a separate database (not shown) contain- 
ing registration data. The registration data contains in- 
formation about databases 122, 124 and 126 (FIG. 1) 
of subscription system 104 (FIG. 1 ). The registration da- 
ta also contains predetermined or sample query de- 
scriptions that can be used by query manager to con- 
struct queries appropriate for each of subscribing data- 
bases 122, 124 and 126 (FIG. 1). 

In step 312, query manager 202 (FIG. 2) generates 
a database query for each of databases 122, 124 and 
126 (FIG. 2) in subscription system 104 (FIG. 1) based 
on the retrieved registration data. The database queries, 
for example, is constructed in the standard Structured 
Query Language ("SQL") designed for relational data- 
base. 

Using the subscriber database information, query 
manager 202 (FIG. 2) performs query translations of a 



general query received from primary database engine 
110 (FIG. 1 ) toa query that is recognizable by subscrib- 
ing databases 1 22, 1 24 and 1 26 (Fl G. 1 ). The format for 
translations is defined by subscriber engines 116, 118 
s and 120 (FIG. 1). 

Synchronization manager 204 (FIG. 2) in step 314 
determines whether the query has an immediate mode 
i.e. requiring immediate attention so that the query is 
distributed to subscribing systems 1 04 (FIG. 1 ) prompt- 
ly. 

If not, the synchronization manager 204 (FIG. 2) 
sets up a date and time interval for each query so that 
queue containing such query can be distributed at the 
prespecrfied date and time. 

In step 318, synchronization manager 204 (FIG. 2) 
in step 318 stores the translated queries in update 
queues 206 (FIG. 2) where the query with the immediate 
mode is placed into an immediate queue and the query 
with specified date and time is placed into a deferred 
queue. 

Update queues 206 (FIG. 2) thus fall into any of 
three following categories: immediate queue (V), de- 
ferred queue ("D") and retry queue ("R"). The retry 
queue is set when the acknowledgment from subscrib- 
ing systems (FIG. 1 ) indicates failure in the transmission 
process and described in more details with respect to 
FIG. 5. 

In step 320, synchronization manager 204 (FIG. 2) 
confirms that query has been generated into queues. 

If not, the synchronization manager 204 (FIG. 2) re- 
turns to step 31 2 to place the query into a queue. 

If confirmed, synchronization manager 204 (FIG. 2) 
determines whether there is any more subscribing sys- 
tem 1 04 that needs the query containing the updates in 
primary system 110 (FIG. 1) 

If there is a subscribing system 104 (FIG. 1) that 
needs the update query, then synchronization manager 
204 (FIG. 2) returns to step 310 to generate query for 
the additional subscribing system(FIG. 1) based on the 
registration data specific to the additional subscribing 
system 104 (FIG. 1). 

If there no subscribing system 104 (FIG. 1) requiring 
the updates, synchronization manager (FIG. 2) in step 
324 increases the number of update transactions ("Un") 
by one. 

In step 326, synchronization manager 104 (FIG. 2) 
determines that all of the subscribing systems 1 04 (FIG. 
1 ) that need updates already have translated queries in 
queues waiting to be transmitted to the subscribing sys- 
tems 104 (FIG. 1). 

If not, the synchronization manager 104 (FIG. 2) re- 
turns to step 308 so that query manager 202 (FIG. 2) 
review the updates that were received from primary sys- 
tem 102 (FIG. 2). 

If all queues necessary for updating the subscribing 
systems 104 (FIG. 1) are generated, synchronization 
manager 104 (FIG. 2) in step 328 is triggered to proceed 
with distribution as described in greater detail with re- 
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spect to FIG. 4. 

Referring to FIG. 4, synchronization manager 204 
(FIG. 2), in step 400, constantly checks the status of the 
queries in queues 206 (FIG. 2) that need to be transmit- 
ted. Synchronization manager 204 (FIG. 2) checks the 
queue status by first determining in step 401 whether 
there is any queue in queue controller 208 (FIG. 2> hav- 
ing the immediate mode. 

If there is any queue having the immediate mode, 
synchronization manager 204 (FIG. 2) in step 402 se- 
lects all the queues with the immediate mode and 
groups them for transmission to subscribing systems 
104 (FIG. 1). 

For each of databases 112, 124 and 126 (FIG. 1), 
synchronization manager 204 (FIG. 2) in step 404 proc- 
esses concurrence check by first determining in step 
406 whether the number of update transactions, Un, is 
greater than or equal to one, i.e., whether there is at 
least one updates in the queue. 

if there is at least one update transaction in the 
queue, synchronization manager 204 (FIG. 2) in step 
408 determines if the number of queries subtracted by 
the concurrence level is greater than 0. Synchronization 
manager 204 (Fl G. 2) thus checks whether concurrence 
controller 21 0 (FIG. 2) can handle the number of queries 
handled by query manager 202. 

If it is found that concurrence controller 210 (FIG. 
2) can handle the number of queries handled by query 
manager 202, synchronization manager 204 (FIG. 2) in 
step 410 enables concurrence controller 210 (FIG. 2) to 
open the distribution channels for queues and data dis- 
tributor 212 (FIG. 2) distributes update transaction for- 
matted for each of database engines 116, 118 and 120 
(FIG. 1) of subscribing system 104 (FIG. 1 ) via commu- 
nication network 106 (FIG. 1). 

Synchronization manager 204 (FIG. 2) returns to 
step 400 to determine whether all the queues with the 
immediate mode have been processed. 

If there is no more immediate queue, synchroniza- 
tion manager 204 (FIG. 2) in step 412 selects and 
groups all the retry queues in the update queues 206 
(FIG. 2). 

Synchronization manager 204 (FIG. 2) in step 414 
determines if any retry queue is found from step 412. If 
not, synchronization manager 204 (FIG. 2) proceeds to 
step 422 to process the rest of the queues which would 
be deferred queues at this point. 

If any retry queue is found, synchronization manag- 
er 204 (FIG. 2) in step 416 examines the retry queue by 
verifying that the retry count does not exceed the pre- 
determined maximum count number. 

If the retry count exceeds the predetermined maxi- 
mum number of requeries, synchronization manager 
204 (FIG. 2) prompts in step 418 a notice to a system 
administrator reporting that the retry count exceeds the 
maximum number. This step ensures that the transmis- 
sion resources and queuing resources are not wasted 
on queues that are directed to missing or defective sub- 



scribing systems. 

If the retry count does not exceed the maximum 
number of queries, synchronization manager 204 (FIG. 

2) determines if the date and time attached to the queue 
s falls is valid, i.e. the date and time falls within the specific 

interval allocated for the data distribution. 

If the date and time fails within the specific interval 
allocated for the data distribution, synchronization man- 
ager 204 (FIG. 2) proceeds to step 404 for concurrence 

10 check and subsequent distribution. 

If the data and time does not fall within the specific 
interval allocated for data transmission, synchronization 
manager 204 (FIG. 2) proceeds to step 422 to select 
and group deferred queues. Thereafter synchronization 

15 manager 204 proceeds to step 404 for concurrence 
check and subsequent distribution. 

FIG. 5 illustrates the distribution process to sub- 
scribing systems 104 (FIG. 1) that was discussed in re- 
lationship to step 410 in more details. 

20 in step 500, as data distributor 212 (FIG. 2) trans- 
mits the update transaction query, the status of the 
sending query is logged for records with date and time. 
The update query that is sent to subscribing systems 
104 (FIG. 1 ) requires acknowledgment and prompts da- 

25 tabase engine 116, 118 or 120 (FIG 1) to return the ac- 
knowledgment. The acknowledgment indicates whether 
the subscribing systems have received the transmitted 
query with the following symbols: query update complet- 
ed ("Uc") or query update failed ('UP) . 

30 in step 502, response handler 214 (FIG. 2) receives 
the acknowledgment from database engine 116, 118 or 
120, and logs the acknowledgment. 

In step 504, response handler 214 (FIG. 2) triggers 
synchronization manager 204 (FIG. 2) to maintain cur- 

35 rent information with the acknowledgment. 

Synchronization manager 204 (FIG. 2) in step 506 
determines whether the acknowledgment for each que- 
ry indicates query transmission was successful and 
completed. 

40 |f acknowledgment indicates that query transmis- 
sion is a failure, synchronization manager 204 (FIG. 2) 
in step 508 moves the queue so that such retry queue 
will be picked up for transmission again in step 326 (FIG. 

3) . 

45 |f synchronization manager 204 (FIG. 2) receives 
the acknowledgment indicating successful transmis- 
sion, synchronization manager 204 (FIG. 2) in step 510 
removes the queue from update queue 206 (FIG. 2), and 
decreases the concurrence control level in concurrence 
50 controller 210 (FIG. 2) by one. 

It will be understood that the foregoing is only illus- 
trative of the principles of the invention and that various 
modifications can be made by those skilled in the art 
without departing from the scope and spirit of the inven- 
ts tion. 



5 



9 



EP0 877 323 A2 



10 



Claims 

1 . A database network having a primary database and 
a plurality of heterogeneous subscribing databases 
for replicating data updates of said primary data- 
base in said heterogeneous subscribing databases, 
comprising: 

a database engine connected to said primary 
database for capturing said data updates in 
said primary database; 

a query manager connected to said database 
engine for generating queries, each of said 
queries being translated based on a specified 
format for each of said heterogeneous sub- 
scribing databases; and 
a data distributor connected to said query man- 
ager for distributing said translated queries to 
said heterogeneous subscribing databases. 

2. The database network in claim 1, further compris- 
ing: 



and 

said synchronization manager for redistributing 
said queries if said updates in said subscribing 
databases fail. 

5 

8. A method of replicating data updates of a primary 
database in a plurality of heterogeneous subscrib- 
ing databases in a database network having said 
primary database and said heterogeneous sub- 

10 scribing databases, comprising: 

capturing said data updates in said primary da- 
tabase; 

generating queries, each of said queries being 
15 translated based on a specified format for each 

of said heterogeneous subscribing databases; 
and 

distributing said translated queries to said het- 
erogeneous subscribing databases. 

20 

9. The method in claim 8, further comprising the steps 

of: 



a subscription controller for verifying that at 
least one of said subscribing databases needs 
said updates; and 

said query manager for generating queries only 
for said at least one of said subscribing data- 
bases that needs said updates. 

3. The database network in claim 1 , further comprising 
a synchronization manager connected to said query 
manager for controlling said query distribution. 

4. The database network in claim 3, further comprising 
a plurality of update queues connected to said que- 
ry manager for storing said queries generated from 
said query manager. 

5. The database network in claim 4, further comprising 
a queue controller connected to said update queues 
for prioritizing said queues in distribution. 

6. The database network in claim 5, further compris- 
ing: 

a concurrence controller connected to said 
queue controller for opening distribution chan- 
nels of said distributor; and 
said synchronization manager enables said 
concurrence controller only if said queues can 
be handled by said concurrence controller. 

7. The database network in claim 3, further compris- 
ing: 

a response handler for verifying completion of 
said updates in said subscribing databases; 



verifying that at least one of said subscribing 
£5 databases needs said updates; and 

generating queries only for said at least one of 
said subscribing databases that needs said up- 
dates. 

30 10. The method in claim 8, further comprising the step 
of controlling said query distribution. 

11. The method in claim 10, further comprising the step 
of storing said queries generated from said query 

35 manager. 

12. The method in claim 11 , further comprising the step 
of prioritizing said queues in distribution. 

40 13. The method in claim 12, further comprising the 
steps of: 

opening distribution channels of said distribu- 
tor; and 

45 enabling said distribution channels only if said 

queues can be handled by said distribution 
channels. 

14. The method in claim 10, further comprising the 
so steps of: 

verifying completion of said updates in said 
subscribing databases; and 
redistributing said queries if said updates in 
55 said subscribing databases fail. 
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